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ABSTRACT 



Identifying the growing interest in early childhood programs 
and services, this paper offers practical guidelines for collecting accurate 
and useful information about early childhood services in a particular country 
or region, focusing on data collection and system monitoring. The paper 
discusses how. programs develop out of policy decisions and how monitoring 
practices can be tailored to be consistent with state policies. Examples are 
provided of data collection methods used to examine program coverage and 
quality, two areas especially important for policy decisions. The paper also 
discusses unique demands of data collection for cross-national research, and 
examines the differences between the types of programs falling under early 
childhood services in developed and developing countries. Three types of data 
collection are described: program questionnaires, household surveys, and 
special studies of programs and children. The use of these methods for system 
monitoring is explained, and the use of a more general procedure integrating 
these methods and using the findings for future planning is discussed. The 
paper presents methods to collect data regarding coverage/equity and program 
quality for countries focusing on center-based programs and those focusing on 
non-center-based programs. Guidelines for collecting data for cross-national 
purposes are presented and findings from cross— national studies are detailed. 
The paper concludes by asserting that a careful approach to research design 
can yield accurate and useful information even in challenging situations. The 
following principles for a systematic approach to data collection and system 
monitoring conclude the report: (1) carefully define the type of service to 

be studied; (2) operationalize outcomes to be measured based on policy goals; 
(3) conform data collection methods and sampling procedures to t he study's 

Reproductions supplied by EDRS are the best that can be made 
from the original document. — 



goals; and (4) pilot test data collection instruments and train data 
collectors to ensure the highest reliability. (Contains 14 references.) (KB) 
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Data Collection and System Monitoring in Early Childhood Programs 



Introduction 

The increasing attention to research on early brain development and on short- and long- 
term effects of children’s participation in quality early childhood programs, along with 
the growing governmental commitment to lifelong learning in the Education for All 
conferences (1990, 2000), has led to growing interest in early childhood programs and 
services in countries around the world (Myers, 2000; UNESCO, 2000). 

The purpose of this paper is to offer practical guidelines to those interested in 
collecting accurate and useful information about early childhood services in a particular 
country or region. The guidelines presented focus on two areas of particular concern for 
policy-makers: data collection and system monitoring. To accommodate the needs of 
both developing and developed nations, we offer a number of different approaches to 
data collection and monitoring. We discuss how programs develop out of policy 
decisions and how to tailor monitoring practices to be consistent with stated policies. 
We provide practical examples of data collection methods that can be used to examine 
coverage and quality, two areas that are especially important for policy- making 
decisions. Finally, we discuss the unique demands of collecting data to be used in cross- 
national research. 

This paper begins with a brief introduction that includes examples of the major 
types of early childhood services in both developed and developing countries. The next 
section of the paper presents general information about early childhood policies, 
indicators, and methods of data collection. This is followed by a presentation of two 
specific examples of early childhood policies, with suggested indicators for these 
policies and recommended methods of data collection. The next section of the paper 
discusses the special requirements of national data collection if the findings are to be 
reported cross-nationally. 




Early Childhood Services: What Are They? 

At meetings of early childhood people from different countries the term early childhood 
services is often heard as though it has the same meaning across countries, but this is 
not the case. For example, there are major differences between the types of early 
childhood services provided in the majority of developed countries and those provided 
in the majority of developing countries. 

In a recent OECD report of a 12-country policy review, Starting Strong: Early 
Childhood Education and Care (OECD, 200 1 ), the authors note that in most developed 
countries, the early childhood situations for children under 3 years of age differ from 
those for children between the age of 3 and entry into formal schooling. Due to parental 
leave policies or cash benefit policies in many developed countries, there is less need 
for services for the children under age 3. Hence, few national-gover nm ent programs 
exist for this age group (and little national data exists), though there is still some need 
for services. The programs that do exist operate at the local level and are few in number. 
In many developed nations, parents generally utilize these local programs and/or other 
informal services (e.g., care by relatives). As the OECD report notes, however, the 
availability of informal services is decreasing as persons who used to provide internal 
child care join the general labor force. In addition, in future years a greater number of 
mothers may return to the labor force sooner to minimize disruptions in their 
professional careers. Consequently, there may be a need to increase services for 
children under age 3 in developed countries. 
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The OECD report, which focused on children from age 3 to school entry, notes 
that many developed countries are close to, or are striving for, universal access to 
services for this age group. And most of these services are provided in group settings. 
The authors state, “[In this study, ]the term early childhood education and care (ECEC) 
included all arrangements providing care and education for children under compulsory 
school age, regardless of setting, funding, opening hours, or programme content”(p. 14). 
The OECD review chose to focus “on organized ECEC provisions in centres and in 
group settings (including schools) and family day care (individuals who provide care to 
non-related children in the carer’s home)” (p. 15). The countries participating in this 
policy review were Australia, Belgium (French and Flemish communities), the Czech 
Republic, Denmark, Finland, Italy, the Netherlands, Norway, Portugal, Sweden, the 
United Kingdom, and the United States. 

These same types of early childhood services are the ones included in the IEA 
Preprimary Project, a 15-nation study of children’s experiences in early childhood 
programs being conducted by the High/Scope Educational Research Foundation. The 
study has been conducted in 9 developed nations (Belgium [Fr.], Finland, Greece, Hong 
Kong, Ireland, Italy, Slovenia, Spain, and the United States) and 6 developing nations 
(China, Indonesia, Nigeria, Poland, Romania, Thailand). The study is cooperatively 
conducted by 15 research coordinators from these countries, all of whom are 
experienced early childhood researchers familiar with the services in their countries. 
The group jointly chose to include the following types of group settings in the study: 
early childhood education or care settings and family day care homes. Thus, both care 
and education settings operating in a center, school, or family day care home were 
included. 

Some international organizations that work with developing nations in operating 
and expanding early childhood services have chosen to use the more general term early 
childhood development (ECD) in place of early childhood education and care. A recent 
UNICEF report. The State of the World's Children 2001 (UNICEF, 2001), gives this 
definition of ECD: “The acronym ECD refers to a comprehensive approach to policies 
and programmes for children from birth to eight years of age, their parents and 
caregivers. Its purpose is to protect the child’s rights to develop his or her full cognitive, 
emotional, social and physical potential. Community-based services that meet the needs 
of infants and young children arc vital to ECD and they should include attention to 
health, nutrition, education and water and environmental sanitation in homes and 
communities” (p. 17). As this definition indicates, the types of early childhood services 
needing attention in developing nations are often more numerous than those in most 
developed nations, often comprising health, nutrition, parenting, and similar services in 
addition to education and care services. As the definition also indicates, these services 
are delivered using a variety of systems, including but not limited to home visits, group 
meetings, and programs broadcast over radio and television stations. Some developing 
countries have ECD programs currently in operation, while others are still designing 
programs or are in the early stages of operating them. At a recent conference (Early 
Childhood Education and Care Conference, 13-15 June, 2001 OECD Conference in 
Stockholm Sweden), UNESCO worked closely with the following developing countries 
on early indicators for ECD programs in their nations: Bhutan, Brazil, Cambodia, Chile, 
China, The Gambia, Kenya, Senegal, Uganda, and Vietnam. 

Reports from the five African countries (The Gambia, Kenya, Malawi, Senegal, 
and Uganda) have been prepared and included as an addition to this paper. All five 
countries were invited to the meeting in Sweden, but only four were able to attend. Each 
report includes (1) information from general sites to provide background data about the 
country and (2) information contributed at the meeting. The Gambia provided copies of 
several instruments they use to gain information about the persons in a home. These 
instruments have been reproduced and are included as extra materials to this paper. 

Examples of ECD services (or programs) currently operating in three developing 
nations are given below to provide information about the variety of content of such 
services and the delivery systems used. 
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Kazakhstan 

Kindergartens (1-7 years of age - education, nutrition, health monitoring) 
Preschools (5-7 years of age - education) 

Prenatal services (preventive health care and parental education) 

Infant care (0-5 years of age - immunization, health information) 

Support programs for children with special needs (health monitoring, life skills) 

Viet Nam 

Day-care Centres (0-2 years of age - includes creches, kindergartens, and home- 
based day care centers) 

Preschools (3-5 years of age - includes kindergartens, private kindergartens, and 
5-year-old classes) 

Kenya 

ECD programs (0-8 years of age) 

Control of diarrheal diseases (0-5 years of age) 

Acute respiratory infections program (0-5 years of age) 

Immunization programs (0-5 years of age) 



To summarize, in developed countries situations for children under age 3 differ 
from those for children between age 3 and school-entry age. There are few services 
available for children under 3, and families generally make their own child care 
arrangements. In nearly all of these same countries, every child between the age of 3 
and school entry has a legal right to ECEC services regardless of the working status of 
the parents, location of residence, ethnic group, and so forth. The countries included in 
the recent OECD report, as well as most other developed countries, provide center - 
based early childhood services for children between age 3 and school-entry age. A very 
small number of these countries may also utilize home visit programs or parent-chilci 
meetings as methods of delivery, but the number is very small, and data about these 
informal services is rarely (if ever) reported. Health and nutritional services for 
children between birth and age of entry into regular schooling are generally available in 
each of these developed countries for most children under a variety of programs. 

Many developing countries need to be concerned with early childhood services 
that differ from those in developed nations in two major ways. First, the types of 
services needing to be offered may be much broader; for example, they may include 
health and nutrition services as well as care and education services. Second, the delivery 
methods may need to be more diverse. While a developing country may choose to have 
young children attend early childhood settings, in some countries it is common to 
instead or also provide services through home visits, mothers’ meetings, and even radio 
programs. While many developing countries offer mainly non-center-based early 
childhood services (to serve, for example, young children living in mountainous regions 
or on islands), some developing nations are striving to provide center-based services for 
all or at least the majority of their young children. Consequently, when the term center- 
based countries is used in this paper, it refers mainly to developed countries, but it also 
includes a few developing nations. 
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Early Childhood Policies, Indicators, and Data Collection Methods - 
General Information 

Early Childhood Policies and Associated Indicators 

Early childhood policy statements set forth what a government hopes will be the 
outcomes of its programs for young children. Sometimes, but not always, the policy 
statement includes guidelines for the development of these programs. A general policy 
statement developed by a nation might read as follows: To provide programs for young 
children to enable them to develop into productive, responsible adults. 

The next step is to describe the specific types of early childhood programs that 
will be developed to achieve the goals of the policy statement. These statements, 
sometimes labeled as either “policy areas” or “general indicators,” serve to make 
specific elements of the general early childhood policy statement more concrete. That 
is, they provide information about what is really meant by the policy statement. For 
example, there are numerous ways to attain the goal stated in the policy statement 
above, and a listing of either policy areas for focus or general indicators identifies the 
components that the government designates as most important. 

Policy areas or general indicators used with the overall policy statement presented 
here might be worded as follows: 

■ Provide early childhood services to all children regardless of ethnicity, 
income, or other characteristics 

■ Provide early childhood services through a variety of delivery methods to 
ensure that these services reach all preschool-aged children 

■ Involve all relevant ministries and agencies in the development and operation 
of these early childhood programs 

■ Design early childhood programs that will both respect the diversity of ethnic 
groups in our country and encourage these groups to work together for the 
common good 

■ Design early childhood programs that will provide a smooth transition for 
children moving into the formal educational system 

■ Develop early childhood programs in conjunction with parents and in which 
parents have major responsibilities at all levels of planning and operation 

■ Develop early childhood programs that are effective 

Although a nation may list more items than these as policy areas, or general 
indicators, this list illustrates how the nation’s policy priorities might be clarified. 

Evans, with Myers, & Ilfeld, (2000) defines an indicator as . . . “a specific 
behavior or result that you can use as a marker to point out how well objectives are 
being achieved” (p. 278). While the general indicators presented in the preceding list 
provide some concrete information to clarify the general policy statement, it is still 
necessary to carry the clarification process one more step in order for the indicators to 
meet the conditions stated in Evan’s definition, i.e., to use them as markers. We will 
refer to this last step in the process as operationalising the indicators, which essentially 
means providing specific targets for each policy area or indicator. Two examples of this 
operationalisation are given here, using indicators from the list above. In keeping with 
the focus of this paper on data collection methods, I have selected the first and last 
indicators for this operationalisation. 

Provide services to all children regardless of ethnicity, income, or other characteristics 

This indicator states that the programs for children in the age-range designated by the 
government (e.g., birth to age 6, age 3 to school-entry age) will be available to all 
children residing in the country. An operationalisation of this indicator could be stated 
as follows: / 00% of the children of each age included in the age-range served by the 
programs will participate in the early childhood program. While this may be the final 
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goal of the program, interim goals may be set with lower percentages to monitor 
progress toward 100%. For example, a goal of 30% coverage might be set for the end of 
the first 3 years of operation. Or, the goals may be operationalised by setting different 
goals for different groups of children: 30% for urban children, 20% for rural children 
(since rural programs may be more difficult to develop and operate). Regardless of the 
actual percentages used, it is important to have specific numerical goals against which 
to measure the results of data collection for monitoring. 

Develop early childhood programs that are effective 

First, it is important to determine what types of effects are of interest and to determine 
when these effects should be measured (e.g., better health status in 6 months, still in 
school at end of third grade). Then numerical goals need to be specified and stated. For 
example, if the current percentage of children who remain in school through third grade 
is 40% (without early childhood programs in operation), the government might wish to 
increase this percentage to 80%, setting interim percentages as goals during the early 
years of program operation. An interim goal of 60% might be established and used to 
assess progress after the first 3 years of program operation. 

Data Collection Methods 

In the field of early childhood services, there are three major types of data collection 
that are presently used to obtain information: program questionnaires, household 
surveys, and special studies of programs and children. 

Program Questionnaires 

In most countries the program questionnaire is the major form of data collection for 
early childhood programs, regardless of whether they are early childhood centers or 
home visit programs. In countries delivering services largely through centers, the 
director or a teacher in each early childhood center is asked to complete this 
questionnaire annually. Typical items on the questionnaire cover the number and 
characteristics of the children enrolled in the center or the structural characteristics of 
the center, such as availability of equipment and materials or of an outdoor play area. In 
nations using home visits as the major delivery form, coordinators of home visit 
programs may be asked to complete this questionnaire annually. Items on a such a 
questionnaire may include the number of home visitors, the number and characteristics 
of the homes/families being visited (e.g., ethnicity, income category), the frequency of 
home visits, and so forth. 

In center-based program countries, the government can compile the data from the 
set of questionnaires to provide information about the total number of centers in the 
nation, the number of children enrolled in these centers, and so forth. Monitoring would 
involve using the questionnaires submitted over a period of time and comparing the 
findings across the set of yearly questionnaires. For example, over a 5-year period of 
time, the findings may show an increase in the number of children attending early 
childhood centers, perhaps due to an increase in the number of centers in operation; or 
the findings may show a decrease in the number of children, perhaps due to a lowering 
of the age of entry into formal schooling or the closing of some centers for financial 
reasons. 

In countries using home visits and other forms of delivery, the government can 
compile the home visit program questionnaires to determine the total number of homes 
served, areas in which the programs operate, characteristics of the homes visited, and so 
forth. Monitoring would entail examining the findings from the questionnaires 
submitted annually over a 5-year period to gain information about the increase or 
decrease in the number of homes visited each year, changes in the training provided to 
home visitors, the addition or deletion of home visit programs in specific areas of the 
country, and so forth. 
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There are three important considerations when using program questionnaires as a 
data collection method in both center-based and non-center-based nations. First, in 
center-based nations, it is critical to distribute the questionnaires to all centers offering 
early childhood programs. Sometimes countries report national figures for enrollment, 
and then add a footnote stating that the figures include only public centers or centers 
operated by the government. When this occurs, the figures do not represent total 
enrollment in early childhood programs. Nations often need to find a way to collect 
information from non-public centers, such as privately operated ones, religious-based 
ones, and so forth. [Note: If early childhood settings in a country include family day 
care homes, then program questionnaires must also be collected from these settings.] 

One possible solution to this problem is for the government to institute a 
registration system for all types of non-government early childhood centers operating in 
the country. This registration system can consist of a form requesting basic information 
about the center (e.g., name, location, operating person/agency, number of children 
served, and so forth). Completion of this form can be a requirement for operating a non- 
government center in the country. This registration system has worked in some 
countries where there is no charge for registering a center and registration does not 
imply government supervision. Such a system provides governments with basic 
information about all early childhood centers in the country, enabling them to report 
more-accurate national findings. 

A second consideration when using program questionnaires is that, in either 
developed or developing nations, it is critical to have a high return rate. That is, it is 
important to have 1 00% of the total number of centers or home visit programs return the 
questionnaire. Only with this high return rate can you be assured that the compiled 
report of findings is accurate. If the return rate is low in developed nations, it may 
indicate that large percentages of specific types of centers have not completed and 
returned the questionnaires (e.g., rural centers, religious-based centers), and thus the 
final figures understate the findings for certain categories of centers. In developing 
nations, a return rate lower than 80% can mean that large percentages of home visit 
programs located in specific areas of the country have not completed and submitted the 
questionnaires (e.g., mountainous areas, island areas). To achieve a 100% return rate it 
may be necessary to use a follow-up letter or telephone call to a specific center or home 
visit program to remind the director/coordinator to complete and submit the 
questionnaire. This entails time and labor, but the quality of the final data is well worth 
the effort. 

A third consideration when using program questionnaires is that, if a nation plans 
to use the results to estimate a coverage percentage, it is critical to have good estimates 
of two numbers. (1) The government must have a good estimate of die number of 
children enrolled in early childhood centers and/or of the children whose families are 
participating in home visit programs. This means the government must distribute the 
questionnaires to all centers or home visit programs and that the response rates must be 
80% or higher. (2) It is critical to have a good estimate of the total number of children 
of specific ages (e.g., 4- to 5-year olds) in the country. This estimate could come from a 
recent population census or from a recent large-scale survey (e.g., health survey) 
conducted in the nation. That the census or survey be recent is particularly important in 
countries with high birth rates or with high rates of immigration/emigration. Only when 
a country has good estimates of both numbers can an estimate of coverage using data 
from the program questionnaires be calculated. If either of the estimates does not meet 
the basic requirements for “good estimates,” then the country is advised to seriously 
consider conducting a household survey, the method of data collection presented next. 

Household Surveys 

A household survey is the recommended data collection method for gathering data from 
a sample of families with preschool-aged children. If the sample is carefully selected, 
the findings can represent the entire population of families with young children living in 
the nation. For all countries, this is the recommended method of data collection to 
obtain estimates of coverage percentages. It may be possible for a country to use (1) 
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enrollment numbers from program questionnaires and (2) a good recent estimate of the 
total population of children in a specific age-range to calculate the percentage of 
coverage. However, in many nations the estimates for one or both of these types of 
numbers is lacking in quality, and therefore a household survey is the only way to 
obtain a good estimate of coverage. This may be particularly true when early childhood 
services are delivered by informal means (i.e., home visits, radio programs) and the 
agencies do not maintain good records that would identify children whose families 
receive two or more informal services simultaneously. In this situation, the overall 
coverage figure would be an overestimate due to the double-counting of some families 
participating in two or more programs (i.e., they might be counted in each program’s 
enrollment data). 

The first step in using the household survey method is to determine the types of 
families to be included in the data gathering (e.g., all families with children from birth 
to school-entry age, all families with children from age 3 to school entry). Then a 
sample of these families is selected by following standard survey sampling guidelines. 
Government personnel in your country who have conducted either a population census 
or a recent large-scale survey in an area such as health or economics can provide 
technical assistance in this area. A household interview is developed consisting of the 
set of questions to be used for reporting the findings (e.g., Are you receiving 
educational home visits? Have you attended any parent discussions at the local health 
facility? and so forth). 

It is important to pilot-test the interview to ensure that the questions can be easily 
understood by a typical parent. Then interviewers are screened, hired, and trained to 
administer the interview. This training should consist of the following steps: (1) discuss 
the purpose of each question with the interviewers, (2) have each interviewer conduct a 
few practice interviews with friends or neighbors, and (3) have each interviewer 
conduct one or two test interviews in the presence of the trainer. This training will 
increase the likelihood that different interviewers will collect data similarly, which 
increases the quality of the data collected. After the interviews are completed, the data 
are coded and analyzed and the report is prepared. 

While this method of data collection is generally used to obtain coverage findings, 
the same set of data can also be used as a source of equity findings. If selected properly, 
the sample will contain the same proportion of families in each subgroup present in the 
population (e.g., urban, rural, various ethnic groups); thus, the data can be examined 
separately for these subgroups to determine coverage differences. This entails a process 
of disaggregating the total set of findings into sets for various subgroups. 

If some subgroups are present in relatively small numbers in the population (e.g., 
families from certain ethnic groups), it may be necessary to oversample this group to 
provide a sufficient number of interviews for the report of disaggregated findings. 
Oversampling means selecting a larger number of this group of families for 
interviewing than would be selected in a regular survey sampling. The technical 
consultant from the population census office can assist with decisions about the need to 
oversample and give guidelines for procedures. 

In some countries it is possible to coordinate an early childhood household survey 
with either a population census or another large-scale survey. This will often result in 
significant cost savings for sample selection and data collection. Sometimes the federal 
Office of Statistics in a country will have a trained staff of household interviewers who 
can be employed to collect the data or who can train the early childhood survey 
interviewers. For some early childhood household surveys it may be possible to use 
program staff (i.e., home visitors) as data collectors. These staff members are skilled in 
relating to families in their homes and will probably be familiar with many of the 
families selected in the sample for the survey. If staff members are used, it is still very 
important to train them for their role in the household survey. This is necessary, since 
conducting interviews in a household survey is a very different task from doing a home 
visit with a family. 
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Monitoring with this method of data collection might entail conducting a 
household survey to obtain baseline data and then conducting another household survey 
at a later time to assess changes. In countries attempting to increase coverage rapidly, it 
may be desirable to conduct a second household survey in 3 to 5 years. In other 
countries it may be sufficient to repeat the survey every 10 years to obtain periodic sets 
of findings on such program aspects as coverage and equity. 

Special Studies of Programs and Children 

Special studies of programs and special studies of children are combined here because 
they both use the same methodology, that is, the selection of a representative sample 
from the population of either programs or children. This category of studies is an 
important one because the findings found in a study with a sample of programs or 
children can be generalized to the total set of programs or children. In general, program 
studies are used to explore what is actually happening in early childhood programs, 
either centers or home visit programs, whereas child studies are used either to 
investigate the behaviors and activities of children in these two major types of programs 
or to evaluate program effectiveness. 

The first step in a special study of programs is to have government personnel or 
program staff prepare a complete listing of the programs (centers, home visit programs) 
for which findings are desired. This might include a listing of all early childhood 
centers in a developed nation or a list_of homes receiving home visits in a developing 
country. In the latter case, this list might be assembled from the records of individual 
home visitors or from the findings of the household survey used to assess coverage. 
Next, standard sampling procedures are used to select a representative sample of these 
programs. In some cases, the sample may be drawn directly from the complete listing, 
while in other cases, geographical areas may first be identified and then programs 
selected within these areas. The final step is to collect the data in the selected programs 
using observation systems, questionnaires, or other procedures. Examples of special 
studies of programs are given in the next section of this paper as a method of data 
collection for assessing the quality of early childhood programs. 

Like special studies of programs, special studies of children are useful when 
certain types of findings are desired but collecting data on all children participating in 
early childhood programs is impossible. An example of a type of study for which this 
method of data collection would be ideal is a study of later effects of early childhood 
programs. In this study, a sample of children is selected using standard procedures, and 
then the data are collected and analyzed. If the sampling is done properly, the findings 
from the sample can be generalized to the total population of children participating in 
the programs. Sample selection can be achieved either directly from a listing of children 
participating in early childhood programs (e.g., those attending centers, those families 
who receive home visits) or by using a two-stage sampling procedure. In the latter 
procedure, a sample of programs is selected in the first stage, and children from those 
programs are selected in the second stage. 

It is possible to combine a special study of programs with a special study of 
children by using the two-stage sampling process just described. In this combined study, 
data on setting characteristics (e.g., teacher behaviors or home visitor behaviors) can be 
gathered to provide program-level findings, and data on child characteristics (e.g., 
behaviors or developmental status) can be gathered on those children who have been 
chosen from these same settings. This particular combination study may be especially 
useful in conducting research to explore the relationship between a program 
characteristic (e.g., staff-child ratio, home visitor behavior) and a child characteristic 
(e.g., developmental status). 

The use of special studies of programs or of children can be used for monitoring 
purposes. The first study conducted with a sample of programs or children can provide 
baseline data for a total group of programs or children. A second such study could be 
conducted at a later time (e.g., 3 to 5 years later), with its findings being compared to 
the first set of findings to examine changes that may have occurred in such program 
aspects as teacher behaviors or children’s developmental status. 
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Overall System Monitoring 

This section of the paper has included several examples of how to use individual 
methods of data collection for monitoring purposes. However, one can also consider a 
more general type of monitoring that includes integrating the various methods of data 
collection for a country’s early childhood programming and then using the findings for 
future planning. The first step is generally a review of the country’s current early 
childhood programming, including the types of services offered, the proportion of 
children participating in each type, the areas of the nation served by each (e.g., urban, 
rural), and the stage of development of each service (e.g., second year of home visit 
program). These data often come from the most recent program questionnaires and may 
provide only estimates. The second step entails determining what types of data or 
information would be useful in assessing the status of services in terms of such things as 
coverage, quality, and so forth. At these review and planning meetings, government 
personnel and early childhood professionals often not only plan major revisions to the 
early childhood programming but also decide on a schedule for future data collection 
activities to monitor the revised programming. 

One example could be a nation that currently offers early childhood services in 
two forms, through centers and through a home visit program, with the centers primarily 
located in urban areas and the home visits conducted in all areas of the country. When 
the group of government and early childhood persons review the functioning of these 
services, they find they have reasonably complete information about the coverage of the 
centers in urban areas but little information about the coverage of the home visit 
program. Also, they decide it is important to collect information about the quality of the 
home visit program and to begin to gather data about the effectiveness of both the 
centers and the home visit program. 

At the conclusion of the meeting, the participants agree to undertake a program of 
data collection that contains the following components. 

■ Collect a basic set of baseline data to assess the coverage of the home visit 

program 

■ Collect data on the quality of the home visit program 

■ Initiate a study of the short- and long-term effects of both programs 

Basic set of baseline coverage data . To decrease the costs involved in conducting 
a household survey, the decision is made to work with the national office of population 
statistics to explore the possibility of adding a small set of early childhood education 
items to the next mini-census (i.e., data collection of 10% of the population) planned for 
2002. The items will include questions about participation in centers and in the home 
visit program. Those parents who participate in one or both types of programs will be 
asked about their general level of satisfaction and about major problems they have with 
the services. 

Quality of home visit program . A special study of programs is designed to gather 
these data. First, a sample of the homes visited is selected from the total group of homes 
compiled from the records of the home visitors. Then the group of home visit 
supervisors from each area meet to develop and pilot-test a set of items to evaluate the 
visits (e.g., communication style, clarity of information conveyed). Then, for each home 
selected in the sample, the supervisor accompanies the home visitor on the visit to 
gather the data. [Note: It is useful to tell the parent that the supervisor is there to gather 
information to improve the home visits.] 

Short- and long-term effects of the programs . A special study of children is 
planned to collect these data. The household survey is used to identify a sample of 
children who participate in (1) no type of early childhood program, (2) centers only, (3) 
home visits only, and (4) both types of programs. Then for each group of children, 
government research staff can begin a record-keeping project by examining each child’s 
school records for the early elementary grades, recording grade retention data, drop-out 
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data, and so forth. Record keeping can continue for the number of years deemed 
important. 

At the conclusion of the household survey, the government personnel and the 
early childhood professionals can review the data and may make decisions about such 
things as increasing the number of home visitors employed in some areas of the nation 
where travel is difficult and time-consuming (e.g., mountainous areas) and where 
therefore there are fewer home visits completed during a week than anticipated. At the 
conclusion of the study of the quality of the home visits, the group may work with the 
home visit supervisors to revise the training of the home visitors. This example shows 
how information gained through general data collection methods can be used to modify 
a country’s ongoing program of early childhood services. 

In summary, this section has presented general in formation about early childhood 
policies, indicators, and methods of data collection. The section has also provided 
examples of indicators and methods of data collection for developed countries offering 
primarily center-based services and for developing countries offering primarily non- 
center-based services. Finally, the section discussed system monitoring and included an 
example of how data collection can provide information to evaluate the current program 
and guide in modifying that program. The next section will present possible indicators 
and suggested methods of data collection for two specific areas of early childhood 
policy. 



Two Early Childhood Policy Areas with Suggested Methods of Data 
Collection for each 

In this section we present two policy areas — coverage/equity and quality - and provide 
specific examples of the use of methods of data collection for center-based countries 
and for non-center-based countries. 

Coverage/Equity 

This first policy area includes two aspects - coverage and equity. While closely related, 
these two aspects require the development of different indicators and require different 
methods of data collection. In addition, each policy area has somewhat different 
meanings in center-based and non-center-based countries and may require the use of 
different methods of data collection. Consequently, following a general discussion of 
coverage and equity, suggestions for data collection methods will be presented 
separately for center-based countries and non-center-based countries. 

Coverage refers to the level of provision of early childhood services, or more 
concretely, the percentage of children who participate in early childhood services. In 
center-based countries, where early childhood services are mainly provided in centers, 
schools, or family day care homes, the level of provision would concretely translate into 
the percentage of children who attend these settings. In non-center-based countries, 
where the delivery of early childhood services is through a variety of means (home 
visits, group meetings, and so forth) and the category of early childhood services is 
more inclusive (e.g., education services, health services, nutrition services, and so 
forth), the level of provision could be thought of either as the percentage of families 
who receive or participate in these various types of ECD services or as the number of 
children in the homes receiving these services. 

Equity refers to equal access to early childhood services by children or families 
regardless of the gender of child, place of residence (e.g., urban or rural), ethnicity, 
special needs, and so forth. A method of data collection sometimes used to obtain 
information about equity in both center-based and non-center-based countries is to 
compare the percentages of children from different categories (e.g., boys and girls) who 
are enrolled either in early childhood programs in general or in different types of early 
childhood programs (e.g., centers, home visits). These data can often be obtained from 
the program questionnaires. 
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Center-Based Countries 

Based on the information included in the recent report of the OECD 12-country policy 
study (2001), one possible indicator appropriate for many developed countries in the 
area of coverage/equity might be to improve the provision of services for children under 
3. Operationalization of this indicator might result in the following: to increase the 
percentage of children under 3 who participate in ECEC services. 

A second indicator that could be written in some developed nations in the area of 
coverage/equity might be to improve the equity of participation in ECEC services for 
children from immigrant groups or other ethnic minority groups. Operationalization of 
this indicator could be worded as follows: to increase the percentage of children from 
immigrant groups and other ethnic minority groups within the country who participate 
in ECEC services. 

A recommended method of data collection to obtain an initial set of information 
for both of these indicators at the same time would be a household survey. With this 
method, a sample of households containing preschool-aged children is selected and 
interviews are administered. Two sets of questions could be prepared for the interview - 
- one set for households containing children under 3 and one set for households with 
children between 3 years of age and the age of school entry. In households with children 
in both age-ranges, both sets of questions would be asked. 

Parents with children under 3 could be asked about their use or non-use of ECEC 
services for children in this age-range. For parents who are using services, information 
could be obtained about the types of services used, parental satisfaction with these 
services, the types of services they ideally would like to have, whether or not they 
would use services in centers if they were available, and so forth. The information 
obtained from such an interview might assist the government by providing information 
about the percentage of parents who indicate they would use services and the types of 
settings parents desire for their children under 3. For example, if a large percentage of 
parents indicate that they prefer family day care homes for children of this age, the 
government may think about developing a network of such settings, along with a 
training program for the primary adults in the family day care homes and a system to 
license them. If parents are not currently using ECEC services, the reasons for non-use 
could provide valuable information also. For example, parents may have been unable to 
locate services but would use them if they could locate them. Or they may indicate that 
they prefer to care for children of this age themselves. The findings from this household 
survey may be very valuable when a government is considering the provision of 
services for children under 3. 

The set of questions for households containing children between age 3 and the 
school- entry age would be designed to provide equity information. For households of 
nonimmigrant groups and ethnic majority groups, survey findings will indicate the 
proportion of children in this age group attending ECEC settings. The survey findings 
from immigrant-group households and ethnic-minority-group households would 
likewise provide the percentage of children between age 3 and school entry age who 
presently attend such settings. Since the number of households in the sample identified 
as belonging to immigrant groups or ethnic minority groups will likely be smaller than 
the number of households in the major groups, it may be necessary to “oversample” 
these groups. Since most developed countries conduct population censuses regularly 
and the information from these censuses is generally available, oversampling is not a 
difficult task. 

In some countries it may be possible to conduct a household survey by telephone. 
A survey area whose households have a telephone-ownership level of 90% or higher is 
generally a requirement for this methodology. This means 90% in all neighboihoods, 
including those in which immigrant and ethnic majority families reside. 

Monitoring with this method for data collection will entail conducting one 
household survey to provide initial estimates of both the percentage of children under 3 
participating in ECEC services and the percentage of participation for two groups of 
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children in the older age range: (1) children from immigrant groups and ethnic minority 
groups and (2) children from non-immigrant groups and et hn ic majority groups. To 
assess the change in these two indicators over time, it would be necessary to conduct a 
second household survey, possibly 5 years later. 

Non-Center-Based Countries 

As the information in the introduction suggests, compared to most developed countries, 
many non-center-based (or developing) countries need to focus on initiating or 
operating early childhood programs that include a wider variety of services. For 
example, developing countries often need to have services dealing with child health, 
child nutrition, and parent education in addition to care and education services for 
children. Also, in many countries, these services are delivered to the homes through a 
variety of methods such as home visits, community meetings, and radio programs. Since 
many of the services are delivered directly to families, the household survey is the only 
method of data collection that can be used to gather information about coverage and 
equity. 

We will provide two examples of the use of this method of data collection to 
provide coverage and equity findings in a non-center-based country. In the first 
example, the country provides early childhood services through centers and a home visit 
program; in the second example, the country offers these two types of services plus a 
weekly informational radio program for families with young children. In both examples, 
the main question to be answered is this: What is the total percentage of children under 
the age of school entry that receive at least one type of ECD service? In countries using 
primarily informal or non-center-based ECD services, it is often easy to count the 
number of families participating in these services, but it is also important to count the 
total number of children being impacted by the services and to note their ages. Since 
attendance at centers is recorded using the child as the unit of measurement, it is 
important to have participation in the type(s) of informal services in this same unit of 
measurement. Using the child as the unit of measurement assumes that all children in 
the family are at least indirectly impacted by the informal services, an assumption 
generally made by the developers and operators of informal services such as home 
visits. 



Table 1 presents the findings for the first example, a country that uses centers and 
home visit programs as delivery systems for ECD services. The household survey 
interview would have separate questions about the family’s (i.e., children’s) 
participation in each type of program (e.g., center, home visit) to identify those 
participating in only one type and those participating in both types. This set of questions 
allows separate percentages to be calculated for (1) children in centers, (2) children 
receiving home visits, and (3) those participating in both types of services. For both 
examples, we will report the findings for the following age categories of children: under 
3 years old, 3 years old, 4 years old, and 5 years old. 

Table 1 Percentages of children participating in one or more of two types of 
early childhood programs by age of child 



Child’s age in years 



Type of Service 


Under 3 


3 


4 


5 


Center only 


0 


5 


15 


30 


Home visits only 


40 


30 


30 


20 


Center + Home visits both 


0 


10 


15 


20 


% in Center 


0 


15 


30 


50 


% in Home visits 


40 


40 


45 


40 


% in one or both types 


40 


45 


60 


70 



0 
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Table 2 Percentages of children participating in one or more of three types of 
early childhood programs by age of child 



Child’s age in years 



Type of Service 


Under 3 


3 


4 


5 


Center (C) only 


0 


5 


10 


30 


Home visits (H) only 


20 


10 


10 


5 


Radio program (R) only 


30 


25 


20 


5 


Center + Home visits only 


0 


5 


5 


20 


Center + Radio program only 


0 


5 


10 


5 


Home visits + Radio program only 


25 


15 


10 


10 


Center + Home visits + Radio program 


0 


0 


5 


10 


% in Center 


0 


15 


30 


65 


% in Home Visits 


45 


30 


30 


45 


% in Radio program 


55 


45 


45 


30 


% in one or more types 


75 


65 


70 


85 



The findings in Table 1 indicate that no children under 3 years of age attend 
centers, while 50% of children 5 years old do so (30% only attend centers + 20% attend 
centers and receive home visits). Center attendance percentages such as these might be 
expected in a nation that sees centers largely as a means of preparing children for later 
formal schooling and thus focuses more on children closer to school age. For all age 
categories, 40% to 45% of the children participate in home visits only. Finally, if we 
look at the percentage of children participating in one or both types of ECD services, we 
can see that these percentages range from 40% for children under 3 to 70% for children 
aged 5 (e.g., for 4-year-olds, 15% + 30% + 15%). 

Table 2 presents the findings for the second example, in which a nation provides 
three types of ECD services: centers, a home visit program, and a radio program. As in 
example 1, the household survey interview for this situation would contain questions 
about the family’s (children’s) participation in each type of service. From these data it is 
possible to determine the percentage of children participating (1) in just one type of 
service (e.g., home visits only), (2) in only two types of services (e.g., home visits and 
radio programs), and (3) in all three types of services. Using these mutually exclusive 
categories ensures that a child participating in two or more categories will not be 
counted more than once. 

A first look at Table 2 shows that the percentages of children participating in one 
or more services range from 65% for 3-year-olds to 85% for 5-year-olds. However, 
these percentages break down differently by type of program for children of different 
ages. For example, families of under-3s participate mainly in home visit programs and 
in the radio program, while 5-year-old children largely attend centers, with only small 
percentages participating in the home visit and radio programs. 

While Tables 1 and 2 each include only one type of home visit program, it is 
possible to use the household survey to provide separate findings for two or three types 
of home visit programs. This would entail including an interview question for each type 
of home visit and then, assuming it is desirable to report information separately for each 
type, using separate categories for all possible combinations of types of ECD services in 
the results table. Using this procedure for expanding the Table 2 findings to include two 
types of home visit programs would result in the following categories: 
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Single-Type 
Centers (C) 

Home visits #1 (HI) 
Home visits #2 (H2) 
Radio program (R) 



Two-Type-Only 
C + Hl 
C + H2 
C + R 
HI +H2 
HI +R 
H2 + R 



Three-Type-Only 
C + Hl +H2 
C + HI + R 
C + H2 + R 
HI +H2 + R 



All 

C + Hl +H2 + R 



While this may seem to be an unwieldy task, this example illustrates the principle 
underlying the procedure. The task can be simplified by using a computer to conduct 
the analysis and produce such a table. 

Although Tables 1 and 2 present findings for an entire country, it maybe be useful 
to examine the findings for separate regions within a country (e.g., urban areas and rural 
areas). While this disaggregation of the national findings requires a larger sample size, 
it results in high-quality coverage findings for different types of areas. Similar 
disaggregation procedures can be used to examine the coverage findings for children by 
various characteristics (e.g., gender, ethnicity), thus providing equity information. 

Finally, a question can be included about parents’ reasons for nonparticipation in 
specific types of early childhood programs. This would be a follow-up question when 
parents report that they do not participate in a particular program. For example, some 
parents may indicate that they do not participate in home visits because of specific 
characteristics of the home visitor (e.g., gender, ethnic group) or because the 
husband/father does not allow the wife/mother to meet with nonfamily members in the 
home when he is not present, and the home visits are scheduled for those times. Such 
responses can yield important information for future planning purposes. 

Quality 

Quality in early childhood programs is a complex concept influenced by values and 
beliefs and closely bound up with culture and contexts. It is also a dynamic concept, 
with definitions evolving and changing over time. Within this complex, dynamic 
context, it is still possible to identify certain indicators of quality that are of interest to 
most countries. These include input indicators, process indicators, and outcome 
indicators. 

Input indicators, which refer to the structural characteristics of early childhood 
settings or informal services, include such things as training and qualifications of staff 
and availability of materials and equipment. In formal settings, input indicators may 
also include group size and staff-child ratio. Process indicators include the actual 
happenings within a setting or during a home visit. These may include communication 
styles, interpersonal relationships, and the experiences of children in settings (or of 
parents during home visits). Outcome indicators refer to the impact of the program in 
terms of effectiveness and include children’s health, developmental status, and school 
adjustment. 

Operationalized indicators for each of these three categories might be worded as 
follows: 

Input Indicators 

The average staff-child ratio in programs serving 4-year-old children will be 1 staff 
to 10 children. 

At least 70% of the home visitors will have completed a specialized training 
program. 
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Process Indicators 

On at least 70% of the home visits, the home visitor will demonstrate an activity for 
the parent and child to do together. 

In at least 50% of the early childhood centers, children will be interacting with 
other children or with adults at least 15% of the time. 

Outcome Indicators 

- At year’s end, the majority of 4-year-old children who have attended preschool will 
receive a score of 70%> or higher on a developmental status measure. 

By 2006, 70% of the children who have attended early childhood centers will 
complete sixth grade. 

Center-Based Countries 

The literature contains many examples of studies in center-based countries in which the 
data collection focuses on input indicators. The major indicators identified in these 
studies include group size, staff-child ratio, and teacher training and experience. Less 
common indicators include availability of materials and equipment and availability of 
space. The program questionnaire is the data-collection method generally used to gather 
these types of information in most countries. 

Special studies of programs is a recommended method for gathering data on 
process indicators. After selecting a representative sample of settings, data are usually 
obtained by observation. Dining the past several years, different types of observation 
systems have been developed, including the ECERS (Harms, Clifford, and Cryer,1998), 
a rating system that assigns a rating of 1 to 7 for each of 43 separate items, based on a 
minimum of 2 horns observation in a center. The item scores are combined to produce 
an overall score of the setting’s quality. It should be noted that this instrument has been 
developed and primarily used in developed nations. 

A variety of instruments including observation schedules have been used in the 
IEA Preprimary Project, a study of early childhood services in countries on several 
continents. This project was introduced briefly at the beginning of the paper and the 
following is a more detailed description of it. 

Information about Phases 1 and 2 of the IEA Preprimary Project (IEA/PPP) 

IEA Preprimary Project - Phase 1 

In Phase 1 of the IEA/PPP (1987-1989), information about the use of early childhood 
care and education settings has been collected from nationally representative s amp les of 
families in each of the 1 1 participating countries. By a joint decision of the project 
directors in each country, the age-range of the children to be studied by the project was 
3 years 6 months to 4 years 6 months. In most participating countries, this age-range 
took in the child’s last full year before entry into a formal school environment; it was 
also the age-range for which there was the widest variety of early childhood care and 
education settings. 

The 11 countries participating in Phase 1 of the IEA/PPP included Belgium 
(French), China (PRC), Finland, Germany (F.R.G.), Hong Kong, Italy, Nigeria, 
Portugal, Spain, Thailand, and the United States. Phase 1 consisted primarily of a 
household survey in each participating nation to determine the types of early childhood 
care and education services used by families and to deter min e some of the 
characteristics of the families and of the services these families use. Appendix 1 
provides descriptions of the sampling plans used in China (PRC) and Nigeria for this 
study. These sampling plans were developed under the direction of Dr. Leslie Kish 
(1911-2000), a University of Michigan professor emeritus and world-renowned person 
in the field of sampling. Sample items from the Parent/Guardian Interview (P/G), the 
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instrument used in Phase 1 to provide information about the families interviewed are 
presented in Appendix 2 (Olmsted & Weikart, 1994). 

IEA Preprimary Project — Phase 2 

Phase 2 of the IEA/PPP (1991-1993) was a study of the “quality of life” experienced by 
4-year-old children either attending major forms of early childhood setting in their 
countries or spending their time at home. The age-range for the Phase 2 children was 4 
years 3 months to 4 years 9 months. The 1 5 countries participating in Phase 2 included 
Belgium (French), China (PRC), Finland, Greece, Hong Kong, Indonesia, Ireland, Italy, 
Nigeria, Poland, Romania, Slovenia, Spain, Thailand, and the United States. In all 
participating nations in the study, Dr. Kish supervised the sample selection. Appendix 3 
provides examples of the sampling plans for Indonesia and Nigeria for this study. 

The IEA/PPP Phase 2 instruments consisted of three observation systems 
described briefly below, three interviews/questionnaires (Expectations Questionnaire, 
Family Background Interview, and Provider Survey), and five child developmental 
status measures (Cognitive Developmental status, Language Developmental status, 
Fine-Motor Skills measure, Social Competence measure, and Preacademic Skills 
measure). Appendix 4 presents selected items from the Family Background Interview 
for Phase 2 and Appendix 5 presents selected items from the Provider Survey 
Questionnaire (Weikart, 1999). 

Sampling - Phases 1 and 2 

Several sampling plans from Phases 1 and 2 of the IEA/PPP are included in this paper 
to highlight specific features. First, Nigeria is included (both phases) as the only African 
country in the study and to show how the samples were selected for each phase in that 
particular country. Sampling plans from China (PRC) (Phase 1) and Indonesia (Phase 2) 
have been included to show how, in a country that has only “guesses” about the actual 
number of people of each age living in various parts of the country, samples can be 
selected and the study can be conducted. It is possible to conduct a study of early 
childhood education in a country that has limited information about its population, but 
care then needs to be taken in selecting the sampling procedures and in analyzing the 
study data. 

Observation Systems 

A set of three observation systems has been developed by the IEA Preprimary Project 
(IEA/PPP) in conjunction with the 15 -nation Phase 2 IEA/PPP study. Each system 
provides information about a different aspect of early childhood settings, such as 
teacher behaviors or children’s activities. Brief descriptions of the focus of each 
observation system are presented here. 

The Management of Time system records the activities that the adult proposes for 
the children (e.g., physical, preacademic) and the proposed group structure for the 
activity (e.g., whole group, small group). These data are collected continuously during 
two nonconsecutive mornings. 

The Child Activities system records the types of activities (e.g., physical, 
preacademic) in which several individual children engage during a typical morning. 
Also recorded are the children’s interactions with other children and with adults. Using 
a time-sampling methodology, the observer records data for a single child every 30 
seconds for a 10-minute period several times during each morning of observation. 

The Adult Behavior system gathers information about the behaviors of the main 
adult in a setting (e.g., teaching, child management) every 30 seconds throughout a 10- 
minute period several times during each morning of observation. 

The Adult Behavior observation system can be used to gather information about 

behaviors such as the type and frequency of child management behaviors (e.g., 
reminding the child of the rules, giving an order), or the percentage of time that teachers 
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spend listening to children in the setting. These systems have been developed by early 
childhood researchers in 1 5 nations and successfully used in the early childhood centers 
in these countries. The reader is referred to the introduction for a listing of these 
countries (Olmsted, Weikart, and Hayes, in press) 

Special studies of children would be an appropriate method of data collection to 
gather information about the effectiveness of early childhood programs. In a typical 
study, a representative sample of children attending centers is selected, and then data 
are collected either at the end of the program year or at a later time. One example of an 
outcome indicator study that might be conducted is the assessment of cognitive 
developmental status or language skills at the age of formal school entry. Other types of 
outcome indicator data might include determining the percentage of the sample of 
children who are retained or assigned to special education during the early years of 
elementary school, or the percentage of children who stay in formal schooling for at 
least 8 years (i.e.. later-effects study). These latter examples may be especially useful in 
developing countries that provide eail> childhood services primarily through centers. 

Non-Center-Based Countries 

In countries where ECD services are largely delivered through informal means, 
assessing program quality is just as important as it is in countries where services are 
mostly center-based. In this section, we primarily present examples of procedures to 
assess the quality of educational services in non-center-based countries, but we believe 
that parallel procedures for services dealing with health, nutrition, and so forth could be 
developed. 

As in center-based countries, the assessment of input indicators may best be done 
by using data reported in the program questionnaire. Useful indicators might include the 
amount of experience of the home visitors or the amount of specialized training they 
have received. 

Special studies of programs is a recommended method for gathering data on 
process indicators for informal services. For example, program staff would select a 
representative sample of home visitors and then have the home-visitor trainer observe 
each selected visitor making several home visits. Following the home visit, the trainer 
might rate the performance of the home visitor on several variables, such as 
communication style (e.g., treating the parent as an equal, encouraging the parent to do 
activities with the child) and clarity of presentation of ideas. If materials are prepared in 
advance and left with the parents during a home visit, a review of these materials would 
provide information about the content and quality of the home visits. To evaluate the 
early childhood radio programs, a number of the actual programs could be recorded and 
then evaluated with a rating form prepared by senior program staff. Items on this form 
might include clarity of presentation of general information and clarity of instructions 
for specific follow-up activities suggested for parents and children to do together. Also, 
across the total sample of programs, one could evaluate the areas presented, to ensure 
breadth of coverage. 

Findings on outcome indicators for informal programs can be evaluated by special 
studies of children. The first step would be to select a representative sample of children 
in a specific age-range living in homes participating in the home visit programs. Then a 
variety of data could be gathered on the children included in the sample, perhaps by 
home visitors, after they have received special training for the task. For example, there 
may have been a basic hea u h evaluation of children in the home at the beginning of the 
program; another evaluation could be conducted later (e.g., after 2 years of home visits). 
Looking at the “relative” position of the child may provide some data to pvaluate the 
effectiveness of the program in the health area. The data collector may also want to look 
at the child’s immunization record, conduct a short evaluation of the child’s physical 
development (e.g., ask the child to perform simple acts, such as hopping on one foot), or 
administer a set of items to evaluate the child’s language development. Some of the 
outcome indicators included in the section for center-based countries are also 
appropriate here, (e.g., looking at the percentage of children in the home visit sample 
who are retained in the early grades of formal schooling). 
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Another study might use the same sample of children but focus on the home’s 
educational environment. When the home visit program started, the home visitor may 
have gathered information from the parent about what activities the parent and child do 
together or how frequently the parent works with the child on language skills (e.g., 
singing songs, telling stories) or physical skills (e.g., tossing an object back and forth, 
doing simple dances). At a later time, asking the parent the same types of questions may 
provide data to evaluate the effect of the home visit program on the home’s educational 
environment. 

This section of the paper has focused on the policy areas of coverage/equity and 
quality and has presented possible methods of data collection for each of these two 
areas. Up to this point, we have focused on national-level data collection. In the next 
section we will discuss cross-national reporting of data and the special features of 
national-level data collection that allow comparisons of findings across countries. 



Cross-National Data Collection: What Is It and How Is It Done? 

What Is Cross-National Data Collection? 

Cross-national data collection can produce many meaningful findings. If the data 
collected by several countries is to yield meaningful cross-national comparisons, the 
countries need to establish common, specific criteria for category specification, 
sampling, and methods of data gathering/management. Data collection for a cross- 
national table is difficult to plan and execute, but the rewards are often worth the extra 
effort, because this type of table provides each nation with a broader context for 
interpreting its own findings. A cross-national table can provide two types of 
information: First, nations planning to provide new services (e.g., for under-3s) can 
learn which other countries already provide such services and then use these countries 
as information sources. Second, a cross-national table allows each nation to assess how 
its present early childhood services compare with those offered by other countries, 
especially other similar countries (e.g., all developing, non-center-based countries). 

Guidelines for Collecting National Data for Cross-National Purposes 

Using recent reports of early childhood findings (Myers, 2000; OECD, 2000; UNESCO, 
2000), we have used current data to construct Table 3 to provide an example of what a 
cross-national table might look like. With this example we can identify three of the 
major issues in collecting data for a cross-national table and suggest guidelines for 
improving the quality of the findings in such tables. 



Table 3 Percentage of children participating in early childhood programs in 

various countries in 1998 



Part A Developed countries 

Child’s age in years 



Country 


Under 3 


3 


4 


5 


France 


NA 


100 


100 


100 


Greece 


NA 


0 


55 


85 


Italy 


NA 


95 


95 


95 


United Kingdom 


NA 


50 


95 


FS 



ERIC 



23 



Data collection & system monitoring 



Part B Developing countries 



Country 

Cambodia 

Chile 



Kazakhstan 



Child’s age 

3 years to 6 years = 6% 

0 to 5 years = 30% 

0 to 2 years = 4% 

2 years to 3 years = 22% 

3 years to 5 years = 36% 

5 years to 6 years = 83% (1996 estimates) 

2 months to 6 years = 11% 



Kenya 



3 years to 5 years = 35% 



Part C Percentages of children participating in services in various countries 







Child’s age in years 




Country 


Under 3 


3 


4 


5 


France 


NA 


100 (100) 


100 (100) 


100(100) 


China (PRC) 


8(60) 


15 (70) 


35 (75) 


60 (85) 


Country 1 


0(75) 


15 (65) 


30 (70) 


65 (85) 



Code: First percentage = children attending centers, second percentage ( ) = children 
participating in one or more types of early childhood services 

Note. FS = Formal schooling. NA = Not available. 

Meyers (2000), OECD (2000). 



Specifying the categories used in the table 

The most recent cross-national early childhood tables published by OECD (2000) and 
UNESCO (2000) use the following phrase in their titles: “Enrollment rates [or ratios] in 
preprimary education.” From this phrase, one might assume that the data in the table 
reflect the percentage of young children attending early childhood education programs. 
However, as noted earlier in this paper, the OECD countries participating in the recent 
policy study (OECD, 2001) included both education and care services provided in 
centers, suggesting that both services are considered to be important. Also, as noted 
earlier, UNESCO and other international agencies working with countries in the area of 
early childhood services prefer the more general term early childhood development, 
which includes a broad range of services (including, for example, child health and 
parent education services as well as child care and education services). Thus, the 
heading preprimary education services does not seem to be the ideal choice for a table 
of this type containing either OECD or UNESCO countries. 

We propose using the term early childhood programs in the title of a cross- 
national table presenting information about various types of early childhood services. 
The terms center-based services and non-center-based services may be used as the 
labels for subcategories within the table. This system makes it possible to specify the 
types of services provided by individual countries. We propose that government early 
childhood research staff report two percentages; the first would be the percentage of 
children participating in center-based services and the second (in parentheses) would 
be the percentage of children participating in one or more types of service (i.e., center- 
based and/or non-center-based). Thus, the entry for a country would look like this: 35 
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(80), indicating that 35% of children attend centers, and 80% participate in either or 
both types of service. 

Reporting these two types of percentages avoids the problem of over-counting 
children who participate in two or more types of services, a problem that would occur if 
the categories were center-based services and non-center-based services. The data 
presented in Table 1 in the previous section can be used to clarify this point. The 
percentage of children attending centers includes those who only attend centers and 
those who participate in both center- and non-center-based programs: that is, for 3s, 5% 
+ 10% = 15%. If we used the percentage of children only participating in non-center- 
based programs, we would need to include again those children who participate in both 
types of services. Using the percentage of children in any type of service avoids this by 
including the 15% given above plus those children only participating in home visit 
programs (30%) to yield 45%. This 45% is also the sum of the three mutually exclusive 
categories presented at the beginning of Table 1. 

This system of entering two percentages gives equal recognition to different types 
of services. Among countries offering both types of services, comparisons of the 
percentage of children in any type of service may be the more meaningful of the two 
types of comparisons that can be made. At the same time, the table allows countries to 
compare the percentage of children attending centers, which may be the more 
meaningful of the comparisons for countries offering only, or primarily, center-based 
services. 

Part C at the bottom of Table 3 illustrates our proposed entry system for three 
countries and can be used to demonstrate the kinds of comparisons that can be made. 
For example, for 3- year-old children, in both China (PRC) and Country #1 from Table 
3, 15% of the children attend centers, while the comparable percentage for France is 
100%. The second type of comparison reveals that the percentage of children 
participating in one or more types of early childhood services in each country are as 
follows: China (PRC) = 70%, Country #1 = 65%, and France = 100%. These latter 
percentages indicate that in all three countries approximately two thirds or more of 3- 
year-olds are participating in at least one type of early childhood program. 

The second and last term to be clarified for Table 3 is center-based. It is important 
to identify a set of minimum criteria for this term to allow nations to determine when a 
program fits into this category and when it doesn’t. For example, one type of early 
childhood program in Chile is called a Harvest Season Center, in which approximately 
40 children 2-5 years of age are cared for by 2 trained assistants. Would the children 
attending this type of program be included in the center-based percentage or not? Based 
on our experiences with early childhood services in countries around the world, we 
propose that center-based services include those that are in session for 10 hours or more 
per week and for a minimum of 6 months per year. The Harvest Season Centers do not 
meet this criteria for a center-based program, and thus the children participating in them 
would not be included in the center-based percentage in a cross-national table. Our 
criteria are an example only. However, to ensure consistency, there must be some type 
of common criteria that all nations use when preparing data for inclusion in a cross- 
national table. 

Reporting the data by age in years 

Table 3 presents the findings for some countries by age in years (e.g., France) and for 
others by multiage groupings (e.g., Cambodia). Reporting data by the child’s age in 
years is necessary for cross-national comparisons. When a nation reports a single 
percentage for an age-range (e.g., 2 months to 6 years), it is unclear how the provision 
of services is spread across this range. It may be that some nations provide many 
services during the early years of the range (e.g., health and immunization services 
during infancy in Kazakhstan), while others concentrate their services in the year prior 
to formal school entry (e.g., Greece). 

Also, when nations include home visits as a delivery method for services, there 
may be several children in the home receiving these services. For example, a home may 
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include a 2-year-old child and a 5-year-old child, with only the 5-year-old attending a 
center in addition to participating in the home visits. When the data are maintained 
separately by age in years, it is possible to accurately record the types of services 
received by die 2-year-old separately from those by the 5-year-old. 

Specifying the quality of the data collection methodology 

The final major problem to be discussed in compiling findings for cross-national tables 
is the availability of information about the quality of the methodology used to obtain the 
findings. This includes data-gathering quality and data-management quality which are 
important issues. First, we will discuss die method of data collection used to obtain the 
findings. 

There are two major methods of data collection that countries use to produce 
findings for inclusion in a cross-national coverage table. The first method is the 
household survey, described in an earlier section of this paper. As noted there, this 
method is the best one for obtaining coverage data because (1) when data are collected 
from a carefully selected sample of households, the findings can be used to represent 
the total population, (2) a household survey can produce percentages for families 
participating in a single type of early childhood service as well as in all possible 
combinations of services, and (3) it can provide separate findings for children of 
different ages within a single family. Of course, any household survey is only as good 
as the quality of the data-gathering and management procedures followed, but assuming 
these are of good quality, the findings from a household survey can be directly reported 
in a cross-national table. 

The second method often used to obtain data for a cross-national coverage table is 
a program questionnaire. The percentage of children in a particular age-range is 
obtained by compiling the figures across all submitted questionnaires and then dividing 
by the country’s total number of children in that age category (e.g., 4-year-olds) which 
is usually obtained from a recent population census. However, there may be several 
problems with this procedure. (1) This method is best suited to obtaining figures for 
center attendance and less suited to obtaining figures for participation in non-center- 
based programs. (2) As noted earlier, the government may collect program 
questionnaires from only a portion of centers (e.g., government centers), thus providing 
an underestimate of the percentage of children attending centers, or perhaps only a 
small percentage of targeted centers complete and return the questionnaires. (3) The 
population census figure used for the calculation of the coverage percentage may not be 
recent (e.g., more than 10 years old), or there may not even be census data available. 
Considering all of these problems, most coverage percentages calculated from program 
questionnaires must be interpreted with caution. 

With regard to data management, our recommendation is to include all coverage 
percentages submitted by different nations in the cross-national table, but to identify in 
some manner those sets of findings obtained through poor methodology. For example, 
an asterisk could be used to denote findings that need to be interpreted with caution. 
The IEA organization uses this system in its cross-national reports (Mullis, Martin, 
Beaton, Gonzalez, Kelly and Smith, 1997) and it has been positively received by 
readers. In general, assuming a household survey was well conducted, the findings 
would be presented without notation. For findings resulting from the use of program 
questionnaires, a set of two or three questions could be developed requesting 
information such as the types of centers from which questionnaires were obtained (e.g., 
all, or only government centers), the source for the total number of children in a specific 
age-range (e.g., recent population census), and so forth. If the responses to the majority 
of questions are satisfactory, then the findings would be presented without notation. 




Conclusion 

As early childhood services of all types proliferate in both developed and developing 
nations, there is growing recognition of the need to accurately document and monitor 



25 



Early Childhood & Family Policy Series n°5 



such services. While the types of services and available resources vary across countries, 
we suggest that a careful approach to research design can yield accurate and useful 
information in a wide variety of circumstances. Although the tasks present special 
challenges in countries with limited infrastructure and resources, experience has 
demonstrated that it is not impossible. 

In this paper we have presented a systematic approach to data collection and 
system monitoring that can be applied to all research studies, whether they are designed 
to supply information about center-based coverage for 4-year-old children across an 
entire country or to monitor the quality of a home-visit program in a remote rural 
region. The approach is rooted in the following basic principles: 

The type of service to be studied must be carefully defined. ECD services vary 
widely both within and across countries. Many developed countries are most 
interested in information related to center-based services, while developing 
countries often need to study a broader range of services and a more diverse 
means of service delivery. 

Outcomes to be measured must be operationalized based on policy goals. 
Researchers should look first at a country’s ECD policy to determine the goals of 
ECD programs. It is helpful to specify outcomes in terms of measurements that 
can be quantified (e.g., 40 percent of children between the ages of 3 and 5 years 
attend center-based programs). Qualitative data (e.g., asking a parent to state his 
or her opinion about a specific program) can be helpful as well, but is difficult to 
summarize and report when drawn from a large sample. 

Data-collection methods and sampling procedures must conform to the goals of 
the study. If the goal is to collect policy-making information that accurately 
reflects the reality, either all programs or families affected by the policy must be 
included in the data collection, or a representative sample must be selected. To 
obtain the most accurate information, procedures should be put in place to ensure 
a return of 80 percent or higher. 

Data-collection instruments must be pilot-tested and data collectors must be 
trained to ensure the highest reliability. The increase in the usefulness and quality 
of the data gathered far outweighs any costs incurred by putting these procedures 
into practice. 

Cross-national data collection procedures are challenging but have the advantage 
of producing findings that can be meaningfully interpreted within and between 
countries. This type of data collection is difficult because each country’s data collection 
must meet specific requirements with regard to category specification, sampling, and 
data gathering and management. The most useful cross-national findings are produced 
when all of the countries involved cooperate in the planning and design of the research. 
When presenting cross-national findings, special care must be taken in defining 
categories used in tables so they apply equally well to all countries represented. It is 
also especially important to monitor the data-collection procedures and report any 
difficulties that may affect the quality of the findings in a particular country. Cross- 
national research is worth the extra effort, because the findings can enable individual 
nations to compare themselves with other nations and to use these comparisons to 
discern directions for their own future planning regarding early childhood services. 
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